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Abstract 

In this paper we study the fluctuations of the probability distributions of the overlap in mean 
field spin glasses in the presence of a magnetic field on the De Almeida-Thouless line. We find that 
there is a large tail in the left part of the distribution that is dominated by the contributions of 
rare samples. Different techniques are used to examine the data and to stress on different aspects 
of the contribution of rare samples. 



I. INTRODUCTION 



Spin glass models show amazing physical properties. Let us considering for simplicity 
m ean field spin glasses, fike the She.ington-Kfikpatrick m0 del fl, whose solution is given 
by the hierarchical replica symmetry breaking (RSB) Ansatz [3H6(. The model is defined by 
the following Hamiltonian 

%?] = ~ 2 JijWj , (1) 

where <7j = ±1 are N Ising spins and the are quenched random couplings with zero mean 
and variance 1/JV. 

For each sample JT, that is for a choice of the quenched random couplings, one can 
compute the probability distribution function of the overlap, q = X)i=i CiTi/N, between two 
replicas a and f subject to the same Hamiltonian ([1]): we call Pj{q) such a probability 
distribution. 

In the SK model the order parameter in the thermodynamical limit is given by a function 
q(x) : [0, 1] — > [0, 1], related to the probability distribution P(q) of finding two replicas at 
an overlap q, where P(q) is defined as 



P(l) = Pj(q) , 

and the overline represents the average over the samples J^'s. 

The overlap distribution Pj{q) strongly fluctuates from sample to sample. In the low 
temperature spin glass phase (T < T c ), these distributions are not self-averaging, that is 
the typical Pj{q) is very different from the disorder averaged distribution P(q), even in 
thermodynamical limit. The size of these fluctuations in the SK model can be quantified by 
using the Ghirlanda-Guerra relations {3!. [?-[lo|; the simplest identity is 
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Pj(q)Pj(q') - Pj(q) Pj{q') = 3 [Kq - q') - P(q)\P(q') , 

and the r.h.s. is non null as soon as the P(q) is not a delta function, i.e. when replica 
symmetry is broken. 

These large sample-to-sample fluctuations play a very relevant role in numerical simula- 
tions, since they require a huge number of samples to obtain reliable measurements in the 
low temperature phase of spin glass models, and they may produce finite size effects that 
vanish very slowly, increasing system size. 



In the present paper we study overlap distributions in a mean-field spin glass model, 
defined on a Bethe lattice of fixed degree, in the presence of an external field. We focus on 
the data measured at the critical temperature T c , such that the mean overlap distribution in 
the thermodynamical limit is a delta function, P(q) = 5(q — go)- This choice has two main 
advantages: 

• we know analytically the value of T c and qo, by solving the model with the cavity 
method, and this allows us to better study deviation from the thermodynamical limit 
(i.e., finite size effects); 

• the system is critical and so it shows very large sample to sample fluctuations. 

For any temperature different from T c one of the two above statement would be false, thus 
making our study less interesting. Moreover the presence of the external field breaks the 
global spin inversion symmetry and implies that overlaps are non-negative in the thermo- 
dynamical limit: however it is well known that a large tail in the negative overlap region is 
present in systems of finite size and its origin needs to be clarified. 



II. THE MODEL AND THE NUMERICAL SIMULATIONS 



We study an Ising spin glass model defined on a Bethe lattice of fixed connectivity c = 4 
(i.e., a random regular graph of fixed degree c = 4). The Hamiltonian is 

H = -^2 JijOiOj - Hj2 a i , ( 2 ) 

<ij> i 

where <7j = ±1 are N Ising spins, the couplings Jjj = ±1 (with equal probability) are 
quenched random variables and the sum runs over all pairs of neighboring vertices in the 
graph. We use a constant external field H > 0. For not very small connectivity Ising spin 
glasses on a Bethe lattice share many properties with the Sherrrington-Kirkpatrick model 
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121 ] : in the limit c — > oo we recover the Sherrington Kirkpatrick model, and the 1/c 



corrections are well under control 
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We construct the random regular graph in the following way: we attach c legs to each 
vertex and then we recursively join a pair of legs, forming a link, until no legs are left or a 
dead end is reached (this may happen because we avoid self-linking of a vertex and double- 
linking between the same pair of vertices); if a dead end is reached, the whole construction 
is started from scratch. 
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Similarly to the SK model, the model ([2]) has a continuous spin glass phase transition at 
a critical temperature T c which depends both on the value of c and H. At variance with 
the SK model, the critical line in the (T, H) plane does not diverge when T — > 0, but rather 
reaches a finite value H c (see Fig. [Q. This is due to the finite number of neighbors each 
spin has on a random graph of finite mean degree (while this number is divergent with the 
system size in the SK model). In this sense the present model is closer to finite dimensional 
models than the SK model is. 
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FIG. 1. Phase diagram in the temperature-field plane for the J = ±1 spin glass model defined on 
a Bethe lattice of fixed degree c = 4. In this work we report data collected at the critical point 
marked by a big dot. 



The replica symmetric (RS) phase of model fl2]) can be solved analytically by the cavity 
method [12|. In particular one can find the boundary of the RS phase, beyond which the 



model solution spontaneously breaks the replica symmetry [14J, [15]. In Fig. [T] we show such a 
critical line in the (T, H) plane for the model with fixed degree c = 4. The high temperature 
and/or high field region is replica symmetric, while a breaking of the replica symmetry 
is required in the low temperature and low field region. We have checked that the phase 
boundary behaves like H C {T) oc (T — T c ) 3 / 2 close to zero-field critical point T c , and the 
exponent is the same one found in the SK model. 



We have carried our Monte Carlo simulations at the point marked with a big dot in Fig. [TJ 
that is H = 0.7 and T = 0.73536. The uncertainty on the critical temperature for H = 0.7 
is 10 -5 . At that point the value of the thermodynamical overlap is go — 0.67658(1). Please 
notice that we have chosen a rather large value of the external field, which is roughly half 
of the largest critical field value H C (T = 0) ~ 1.53, in order to avoid crossover effects that 
could be due to the vicinity of the zero- field critical point. 




FIG. 2. Disorder averaged overlap probability distributions P(q) show an exponential tail for 
q < Qo- 



Monte Carlo simulations have been performed by using the Metropolis algorithm and the 
parallel tempering method: we used 20 temperatures equally spaced between T max = 2.0 
and T c = 0.73536, and we attempted the swap of configurations at nearest temperatures 
every 30 Monte Carlo sweeps (MCS). Each sample (of any size) has been thermalized for 2 24 
MCS and then 1024 measurements have been taken during other 2 26 MCS: so there are 2 16 
MCS between two successive measurements and we have checked this number to be larger 
than the autocorrelation time. We study systems of sizes ranging from N = 2 6 to N = 2 14 , 
with a number of samples ranging from 5120 for N = 2 6 to 1280 for iV = 2 14 . We are going 
to present only the data for sizes N < 2 12 for which we have simulated at least 2560 samples; 
indeed the data for N = 2 13 and N = 2 14 are more noisy (due to the limited number of 
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samples), moreover we fear that some samples may not be perfectly thermalized even after 
2 26 MCS. By restricting to N < 2 12 we are fully confident about the numerical data. 



III. RESULTS 

We start by showing in Fig. [2] the disorder averaged P(q) for different sizes. The expo- 
nential tail on the left side is evident from the plot (which is on a logarithmic scale): this 
tail goes far in the negative overlap region for small sizes. In the following we are going to 
show that this exponential tail is not a feature of typical samples, but it is completely due 
to very rare and atypical samples. 
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FIG. 3. Mean (left) and variance (right) of P(q). A naive analysis would predict an asymptotic 
value for (q) larger than qo and finite size corrections decaying faster than the iV -1 / 3 expected 
behavior. The variance decays roughly as the expected TV -2 ' 3 law. 



The vertical line at g = go iu Fig- El marks the location of the delta peak in the thermo- 
dynamical limit. By looking at the mean and the variance of P{q) we have checked how 
finite size effects decay to zero. We see in Fig. [3] that while (g 2 ) c decays in a way compat- 
ible with the expected behavior TV -2 / 3 (the discrepancy can be well explained in terms of 
small scaling corrections), the mean overlap (q) shows finite size corrections proportional 
to iV" 1 / 2 (instead of the expected iV^ 1 / 3 ) and seems to extrapolate to a thermodynamical 
limit different from q . This means that a naive extrapolation to the thermodynamical limit 
would produce a wrong estimate of q . The most probable explanation is that finite size 
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corrections of order iV x l 2 have a much larger coefficient than those of order N 1//3 and then 
much larger sizes are needed to observe the asymptotic behavior. 

100 c 1 1 1 q 




FIG. 4. The integrated probability distribution x(q) averaged over the disorder. The value of qo 
can be well determined by the crossing point of data set in the main panel. The inset show the 
scaling function x(N 1 ^ 3 (q — qo)). 



A much better way to estimate go from the disorder averaged data seems to be the analysis 

of the overlap integrated probability function 

f 9 i i 
x(q) = J_ i P(q)dq . 

This variable has been used for studying the behavior of three dimensional systems at zero 
magnetic field [if], [n| ■ The results are shown in Fig. HJ In the main panel we see again the 
exponential tail on the left side, but the crossing point of the functions x(q) estimates with 
a high accuracy the right value for go- in the present case all the crossing points for the sizes 
shown are within a distance less than 10~ 3 from the thermodynamical value and converge 
to it according to the N" 1 ^ 3 law. In the inset of Fig. H] we show that x(q) data perfectly 
collapse when plotted as a function of the scaling variable A^ 1 / 3 (g — q ). 

Let us now turn to the study of sample-to-sample fluctuations. We want to convince 
the reader that the exponential tail is not a feature of typical samples: actually not even a 
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feature of the vast majority of samples, that show roughly Gaussian (or even steeper) tails 
in their Pj(q). The exponential tail is produced by the integration of the secondary peak 
that atypical samples have at an overlap value much smaller than q . 

1 
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FIG. 5. Mean and variance of the 2560 samples of size N = 2 12 . Insets show the overlap probability 
distribution averaged over a small fraction, 1/128, of samples (those in the corresponding circle). 
Solid curves in the insets are Gaussian fits to the data (see text for details). 

Extracting typical and atypical shapes of the Pj(q) from thousands of samples is not 
a straightforward job. We follow the simplest procedure based on the analysis of the first 
two moments. In the main panel of Fig. [5] we show the mean and the variance of the 2560 
samples of size 2 12 . The three insets in Fig. [5] show the averages over 20 Pj{q) chosen 
from typical samples (lower inset) or from atypical samples, either much broader or much 
narrower than typical (upper insets). In every inset we also draw a dashed vertical line to 
mark the location of go- 

We notice that there exist a large difference between typical an atypical samples, both 
quantitatively and qualitatively (especially for the atypical samples showing a double peak 
structure). However the very different shapes can be roughly accounted for by considering 
an effective external field different from the one (H = 0.7 in the present case) appearing in 
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the Hamiltonian: in the atypical samples shown in the upper right inset this effective field is 
larger than H and thus the overlap distribution is narrower and centered on a value greater 
than q Q , while the atypical samples shown in the upper left inset look like if they were below 
the critical line, i.e. with a field smaller than H. 

Since samples with different effective fields will have different critical temperatures, it 
is possible that the main source of sample-to-sample fluctuations can be well described 
by a random temperature (or field) term in the effective Hamiltonian as in the case of 



ferromagnets in random magnetic field 18|-|20|. 



It is also worth noticing that the tails of the distributions shown in the insets of Fig. E]are 
Gaussian or even steeper, as expected . Indeed, the interpolating curves superimposed to 
the bimodal distributions (lower left and upper right insets) have been obtained by assuming 
q = tanh(/t) with a Gaussian distributed local field h. The non-linear transformation is 
necessary (and sufficient) to take into account the small skewness of the distributions. 
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FIG. 6. Most atypical distributions, averaged over a fraction 1/128 of samples: those with the 
largest (left) and smallest (right) variance. By varying the system size they roughly preserve the 
shape and get shrunk according to the scaling q — qo ~ iV -1 / 3 (see insets). 



In Fig. [5] we have presented data only for size N = 2 12 , but a natural question is how 
sample-to-sample fluctuations vary with the system size. We have found that by increasing 
the system size the distribution of the moments shrinks towards the thermodynamical limits 
((?) = Qo an d (q 2 ) c — 0) with the expected iV" 1 / 3 scaling behavior. However it is not true 
that all samples become typical in the thermodynamical limit. In other words the fraction 
of atypical samples (e.g. those with a bimodal distribution) remains roughly constant. In 
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Fig. [6] we show the average P(q) computed on a small fraction (1/128) of samples, those 
most atypical, i.e. those corresponding to upper insets in Fig. [5j We notice that, by varying 
the size, the shape is more or less preserved and the main effect is an overall shrink of the 
distribution. The insets in Fig. [H] show that this shrinking is consistent with the scaling law 
q — Qo ~ 0(A^ -1 / 3 ) that holds at criticality. 
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FIG. 7. Left: Small k moments of Xj(q) measured in systems of size N = 2 12 show a decay faster 
than exponential for q <C qo. Right: Systems of different sizes show the same behavior, once the 
overlap is rescaled accordingly. 




Given that neither typical nor atypical distributions have an exponential tail, the only 
possible explanation is that such a tail is generated by the secondary peak of broader dis- 
tributions when averaging over the samples. We are going to provide quantitative evidence 
for this by looking at the integrated probabilities 

Xj(q) = J^Pj(q')dq'. 

Let us define the moments of the random variable Xj as 

X k {q)=X~^. 

Remind that Xi(q) = x(q) is plotted in Fig. 2] and shows an exponential tail. However in 
the region q <C qo the average Xi(q) is dominated by rare samples, while the vast majority 
of samples has a very small value Xj(q) <C X\(q) and do not contribute to Xi(q). 

In order to extract the behavior of typical samples one should average the random variable 
\og(X j(q)). However for q q there are samples with Xj(q) = and a straightforward 
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FIG. 8. Histograms of Xj at fixed values of X±. The data for X\ = 1/64 has been multiplied by 
10 in order to improve readability. 



computation of log[Xj(g)] is not possible. However, by noticing that 



log[X j{q)] = limlog X k (q) 

k— ¥0 L 



l/k 



it is possible to observe the behavior of typical samples by choosing < < 1. In Fig. [7J 
(left) we plot Xk(q) for k = 1, 1/4, 1/16, 1/64 and we clearly see how the exponential tail 
for k = 1 becomes a Gaussian (or even steeper) decay for k <C 1. Moreover this behavior is 
very well conserved by varying the system size: in Fig. [7J (right) we plot the same averages, 
X\ to Xi/64, as a function of the scaling variables iV 1 / 3 ^ — go) and we see that the data 
collapse (which is very good for q ~ q Q ) remains a reasonable approximation in the entire q 
range. This observation suggests that the entire distribution of the random variable Xj(q) 
mainly depends on the scaling variable iV 1//3 (g — go), or equivalently on the first moment X\. 
This can be checked in Fig. [8] where we plot the distribution of Xj at some fixed value of 
Xi for several system sizes. Please note that the data for Xi = 1/64 have been multiplied 
by a factor 10 in order to avoid overlaps with other data set and improve readability. 

By commenting Fig. [8] we can finally draw the main conclusions of this analysis. First 
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of all, the good data collapse for the probability distribution of Xj at fixed value of X\ is 
a strong indication that we have measured large enough systems in the asymptotic scaling 
regime. Moreover we see that for X\ = 1/2 > x(qo) — 0.429 the distribution of Xj has a 
maximum close to Xi, that is the mean value is representative of typical samples behavior. 
On the contrary, for X\ < x(qo), the distributions of Xj have their maxima at Xj ~ 
and the mean value is not representative of typical values. In particular we observe that for 
very small values of X\ the distribution of Xj develops a power law divergence 1/Xj for 
Xj^O. 



IV. CONCLUSIONS 



In this paper we have seen that on the De Almeida- Thouless line the left tail of the dis- 
tribution of P(q) is dominated by rare samples. The presence of a left tail in the probability 
is q uite an annoying phenomenon that is present also in the Sherrington-Kirpatrick model 



231 ] and in finite dimensional models 24j , both at the phase transition point and below the 



transition. This tail is particularly bothering at not too large magnetic field, because it 
extends in the region of negative q. We think that understanding the origin of this tail may 
be useful in future analysis of the finite dimensional simulations. 

It would be very useful to derive the results on this paper in an analytic way extending 
the techniques of 21] . Indeed in that paper the computation of the tail was done for the 
typical samples and we have to modify it in order to compute the tail of the average over 
the samples. We believe that this is a feasible task. 
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